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Abstract: 

This  paper  describes  the  experimental  apparatus  and  analysis  techniques  now  in 
place  at  the  US  Army  Tank- automotive  and  Armaments  Command  (TACOM)  Visual 
Perception  Lab  in  Warren,  Michigan,  and  the  results  of  some  preliminary  trials  using 
these  methods.  The  tools  will  be  used  to  develop  and  analyze  experiments  to  better 
understand  how  drivers  use  various  displays  and  viewing  systems  intended  to  aid  drivers 
in  poor  visibility  conditions,  particularly  at  night.  Visible  and  infrared  (IR)  road  scene 
imagery  obtained  from  the  field  were  displayed  simultaneously  and  subject  eye  fixation 
and  point  of  gaze  measured  using  a  magnetic  head  tracker  and  near  IR  eyetracker.  Results 
were  analyzed  to  determine  if  personal  differences  between  the  various  subjects  caused 
them  to  prefer  one  display  to  another.  The  goal  was  to  see  if  people  prefer  to  use  the 
heads  up  display  (HUD)  or  the  direct  view  for  driving  at  night. 

Introduction 

As  part  of  a  Cooperative  Research  and  Development  Agreement  (CRADA) 
between  Ford  Motor  Co.  and  the  US  Army  TACOM,  the  investigators  are  doing  research 
on  potential  enhanced  vision  systems  for  automobiles.  The  National  Automotive  Center 
at  TACOM  is  evaluating  novel  display  devices  for  enhanced  vision  in  military  vehicles. 
This  research  parallels  work  in  many  areas  on  the  use  of  enhanced  or  sensor  fused 
imagery  to  aid  in  visual  perception  while  driving.1,2,3 .  The  goal  is  to  develop  various 
metrics  for  the  objective  comparison  of  various  displays  and  viewing  systems  to  aid 
drivers  in  situations  with  poor  visibility.  Specifically,  the  research  is  aimed  at 
determining  what  kind  of  system  would  be  useful  for  assistance  while  driving  at  night. 
One  possible  vision  enhancement  system  uses  an  IR  sensor  to  gather  scene  data  and  a 
HUD  as  a  display  device. 
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Method 

A  modified  F-150  was  used  as  the  test  vehicle  for  this  experiment.  The  F-150 
was  outfitted  with  an  Applied  Science  Laboratories  (ASL)  eyetracker  and  headtracker 
and  a  HUD  supplied  by  Hughes.  For  stimuli,  longwave  infrared  video  and  low-light 
visual  imagery  were  recorded  with  a  HI  8-mm  VCR.  The  high-resolution  video  was 
recorded  at  night  in  various  locations  including  a  highway,  a  city  street,  and  a  country 
road.  The  data  were  recorded  using  an  8  to  12  micron,  uncooled  infrared  camera  and  a 
low-light  visual  camera.  Both  cameras  were  mounted  and  bore-sighted  on  a  High 
Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV).  The  infrared  imagery  was 
displayed  through  the  HUD  mounted  in  the  F-150  test  vehicle,  and  visual  black  and  white 
imagery  was  presented  on  a  large  wrap-around- screen  for  direct  view  through  the 
windshield.  Subjects  were  shown  the  synchronized  HUD  and  direct  view  imagery 
simultaneously  for  a  5-minute  stream  of  video.  They  were  asked  to  look  at  the  scene 
they’d  prefer  to  see  while  driving  during  the  experiment.  The  ASL  eyetracker  recorded 
the  subjects’  point-of-gaze.  Point-of-gaze  is  computed  from  knowledge  of  both  the  head 
position  and  eye  fixation  direction.  The  head  position  and  orientation  of  the  subjects  were 
recorded  using  a  magnetic  head-tracker.  Ordinarily,  a  magnetic  head  tracker  is  difficult  to 
use  inside  a  vehicle  cab  that  contains  magnetic  materials.  However,  if  the  magnetic  pulse 
rate  is  sufficiently  decreased  for  the  eddy  effects  to  decay,  the  measurement  can  be  done 
quite  repeatably.  After  some  experimentation,  it  was  found  that  a  transmitted  pulse 
frequency  of  25  Hz  gave  accurate  results  for  both  head  position  and  orientation. 
Performance  for  the  magnetic  link  was  significantly  improved  by  minimizing  the 
propagation  length.  The  magnetic  transmitter  was  located  close  to  the  receiver  by 
inserting  it  into  the  vehicle  through  the  rear  window.  Figure  1  below  shows  a  picture  of 
the  test  vehicle.  The  infrared  imagery  was  projected  onto  the  HUD  and  the 
corresponding  visual  imagery  was  projected  onto  the  large  180°,  10  ft.  high,  wrap-around 
screen. 


PICTURE  1:  Experimental  F-150  test  vehicle  with  a  subject  wearing  eyetracker 


Data  Analysis 


Figures  1  -  4  are  samples  of  the  fixation  plots  obtained  from  the  subjects.  There  is  an  eye 
fixation  plot  for  each  of  the  subjects  in  the  test.  Distributions  of  fixation  points  for  the 
windshield  and  HUD  are  shown  together  to  represent  the  actual  experimental  geometry. 
There  are  two  areas  of  interest  (AOI)  defined  in  the  eyetracker  data  window.  Area  1  is 
the  small  red  rectangle  and  represents  the  HUD  inside  the  car.  Area  2  is  the  larger  red 
rectangle  and  represents  the  area  of  the  windshield  as  seen  by  a  subject  sitting  in  the 
driver’s  seat  of  the  test  vehicle.  The  small  circles  represent  fixation  periods  that  are 
greater  than  1  second,  and  the  lines  connecting  the  circles  represent  the  movement  of  the 
eye.  The  ASL  eyetracker  recorded  the  subjects’  eye  point-of-gaze  and  the  transition 
sequence  between  points. 

The  subjects  remarked  that  the  image  of  the  infrared  scene  on  the  HUD  was  very  clear 
and  sharp  and  easy  to  look  at  while  in  the  test  vehicle.  In  these  initial  trials  the  HUD  was 
mounted  near  the  preferred  FOV.  In  subsequent  experiments  we  plan  on  moving  the 
HUD  display  to  a  different  location  and  recording  subject  eyetracker  data.  We  shall  also 
turn  the  IR  imagery  source  to  the  HUD  off,  and  record  eye  fixation  data  using  just  the 
visual  imagery. 

The  following  figures  and  tables  summarize  the  data  from  four  initial  subjects.  While  the 
experiments  were  not  yet  designed  to  make  definitive  conclusions,  the  analysis 
demonstrates  the  capability  of  the  experimental  apparatus  and  test  procedures.  In  the 
figures  the  red  spot  in  upper  left  is  the  origin  in  reference  coordinate  system  and  there  are 
three  areas  of  interest  (AOI): 

•  Area  0:  area  outside  the  windshield 

•  Area  1 :  area  inside  windshield,  but  outside  the  HUD 

•  Area  2:  HUD  area 

In  the  tables  below  the  following  terms  are  used: 

•  Dwell  Time  total  time  spent  in  an  AOI 

•  Fix  #  number  of  times  tracked  in  this  area 

•  Mean  sec  average  amount  of  time  in  this  area  per  point-of-gaze 


Tables  1  through  4  are  statistical  analysis  summaries  of  the  probability  of  the  eye  moving 
from  the  HUD  (Area  1)  to  the  windshield  (Area  2)  or  to  the  area  outside  of  windshield 
(Area  0).  The  tables  summarize  the  dwell  time  in  an  area  as  well  as  the  transitions 
between  AOI's.  A  transition  is  defined  as  a  fixation  or  series  of  consecutive  fixations  that 
occurs  in  Area  i  and  the  following  fixation  or  series  of  fixations  occurring  in  Area  j. 


There  are  two  types  of  probability  tables  below,  conditional  and  joint  probability  tables. 
Conditional  probability  is  the  likelihood  that  given  a  fixation  on  one  specific  AOI,  the 
following  fixation  will  be  on  another  specific  AOI, 


Pc(i/j)  = 


No.  of  fixations  from  Af  to  A- 
No.  of  fixations  on  A, 


(1) 


Joint  probability  is  the  relative  likelihood  of  a  transition  between  two  specific  AOI’s  as 
compared  to  transitions  between  other  pairs  of  AOI’s, 


Pj(i/j)  = 


No.  of  fixations  from  Ai  to  Aj 
Total  number  of  transition  pairs 


(2) 


FIGURE  1:  Eyetracker  Fixation  Plot  LCD  Projector,  HUD  Subject  1  IR  &  Visible 


Subject  #  1 

AOI  Dwell  Time 

Fix# 

Mean  Sec 

0 

0 

0 

0 

1 

67 

206 

0.19 

2 

106 

336 

0.28 

Area  Transition 

Area 

0 

1 

2 

0 

0 

0 

0 

1 

0 

144 

205 

2 

0 

205 

335 

Joint  Probability 

Area 

0 

1 

2 

0 

0 

0 

0 

1 

0 

0.43 

0.61 

2 

0 

0.61 

1 

Conditional  Probability 


Area 

0 

1 

2 


0  1  2 

0  0  0 

0  0.7  0.61 

0  0.61  1 


TABLE  1 :  Statistical  Analysis  of  subject  1 


FIGURE  2:  Eyetracker  Fixation  Plot  LCD  Projector,  F1UD  Subject  2  IR  &  Visible 


Subject  #  2 

AOI  Dwell  Time 

Fix# 

Mean  Sec 

0 

0 

0 

0 

1 

43 

75 

0.05 

2 

139 

312 

0.07 

Area  Transition 

Area 

0 

1 

2 

0 

0 

0 

0 

1 

0 

55 

75 

2 

0 

74 

311 

Joint  Probability 

Area 

0 

1 

2 

0 

0.00 

0.00 

0.00 

1 

0.00 

0.18 

0.24 

2 

0.00 

0.24 

1.00 

Conditional  Probability 

Area  0 

1 

2 

0 

0.00 

0.00 

0.00 

1 

0.00 

0.73 

1.00 

2 

0.00 

0.24 

1.00 

TABLE  2:  Statistical  Analysis  of  subject  2 


FIGURE  3:  Eyetracker  Fixation  Plot  LCD  Projector,  HUD  Subject  3  IR  &  Visible 


Subject  # 

3 

AOI  Dwell  Time 

Fix  # 

Mean  Sec 

0 

18.00 

75.00 

0.13 

1 

116.00 

302.00 

0.07 

2 

164.00 

499.00 

0.08 

Area  Transition 

Area 

0 

1 

2 

0 

48 

14 

27 

1 

15 

235 

287 

2 

27 

287 

471 

Joint  Probability 

Area 

0 

1 

2 

0 

0.08 

0.02 

0.05 

1 

0.03 

0.41 

0.52 

2 

0.05 

0.50 

0.82 

Conditional  Probability 

Area 

0 

1 

2 

0 

0.65 

0.19 

0.37 

1 

2 


0.05 

0.05 


0.78 

0.58 


0.95 

0.95 


TABLE  3:  Statistical  Analysis  of  subject  3 


FIGURE  4:  Eyetracker  Fixation  Plot  LCD  Projector,  HUD  Subject  4  IR  &  Visible 


Subject  #  4 

AOI  Dwell  Time  Fix  #  Mean  Sec 


0 

0.65 

4 

2.77 

1 

9.44 

57 

2.06 

2 

33.1 

199 

1.67 

Area  Transition 

Area 

0 

1 

2 

0 

3 

1 

1 

1 

1 

20 

55 

2 

1 

56 

197 

Joint  Probability 

Area 

0 

1 

2 

0 

0.02 

0.01 

0.01 

1 

0.01 

0.10 

0.27 

2 

0.01 

0.28 

0.98 

Conditional  Probability 

Area  0 

1 

2 

0 

1.00 

0.33 

0.33 

1 

0.02 

0.36 

0.98 

2 


0.01 


0.28 


1.00 


TABLE  4:  Statistical  Analysis  of  subject  4 


The  data  from  all  the  subjects  show  they  spent  over  half  their  time  looking  at  the  HUD. 
Although  we  have  over  300  data  points  for  each  subject  we  had  only  4  subjects  for  this 
initial  trial.  The  t-test  and  F-test  analyses  for  mean  fixation  time  shown  below  indicate 
that  there  is  no  strong  difference  in  the  amount  of  time  subjects  looked  at  the  HUD.  This 
is  consistent  with  our  instructions  to  the  subjects  who  received  no  training  and  were 
asked  only  that  they  look  at  whichever  display  they  preferred,  rather  than  perform  a 
driving  task  using  the  IR  imagery. 


AOI  #1 

AOI  #2 

Mean 

0.5925 

Mean 

0.525 

Standard  Error 

0.490142 

Standard  Error 

0.384719 

Median 

0.13 

Median 

0.18 

Mode 

#N/A 

Mode 

#N/A 

Standard  Deviation 

0.980285 

Standard  Deviation 

0.769437 

Sample  Variance 

0.960958 

Sample  Variance 

0.592033 

Kurtosis 

3.917462 

Kurtosis 

3.660675 

Skewness 

1 .976498 

Skewness 

1 .908536 

Range 

2.01 

Range 

1.6 

Minimum 

0.05 

Minimum 

0.07 

Maximum 

2.06 

Maximum 

1.67 

Sum 

2.37 

Sum 

2.1 

TABLE  5:  The  summary  statistical  table  for  mean  fixation  time 


F-test  Two-Sample  for  Variances 


t-test:  Two-Sample  Assuming 


Unequal  Variances 


AOI  #1  AOI  #2 

Variable  1  Variable  2 

Mean 

0.5925  0.525 

Mean 

0.5925  0.525 

Variance 

0.960958  0.5920333 

Variance 

0.960958  0.592033 

Observations 

4  4 

Observations 

4  4 

df 

3  3 

Hypothesized 
Mean  Difference 

0.5 

F 

1.623149 

df 

6 

P(F<=f)  one-tail 

0.350201 

t  Stat 

-0.69412 

F  Critical  one-tail 

9.276619 

P(T<=t)  one-tail 

0.256787 

t  Critical  one-tail 

1.943181 

P(T<=t)  two-tail 

0.513573 

t  Critical  two-tail 

2.446914 

TABLE  6:  f-test  and  t-test  for  mean  time 

AOI  #1 

AOI  #2 

Mean 

160 

Mean 

336.5 

Standard  Error 

57.8172408 

Standard  Error 

61 .85534 

Median 

140.5 

Median 

324 

Mode 

#N/A 

Mode 

#N/A 

Standard  Deviation 

115.6344816 

Standard  Deviation 

123.7107 

Sample  Variance 

13371.33333 

Sample  Variance 

15304.33 

Kurtosis 

-2.764730894 

Kurtosis 

1.515581 

Skewness 

0.540586548 

Skewness 

0.590395 

Range 

245 

Range 

300 

Minimum 

57 

Minimum 

199 

Maximum 

302 

Maximum 

499 

Sum 

640 

Sum 

1346 

Count 

4 

Count 

4 

TABLE  7:  Summary  table  for  number  of  fixations 


t-Test:  Paired  Two  Sample  for  Means 


AOI  #1  AOI  #2 


Mean 

160 

336.5 

Variance 

13371.33 

15304.33 

Observations 

4 

4 

Pearson 

Correlation 

0.91568 

Hypothesized 
Mean  Difference 

0 

df 

3 

tStat 

-7.09175 

P(T<=t)  one-tail 

0.002884 

t  Critical  one-tail 

2.353363 

P(T<=t)  two-tail 

0.005767 

t  Critical  two-tail 

3.182449 

TABLE  8:  t-test  for  number  of  fixations 


It’s  interesting  to  observe  that  there  is  no  significant  difference  between  the  HUD  and 
windshield  for  mean  fixation  times,  however  there  is  a  significant  difference  between 
HUD  and  windshield  for  the  number  of  fixations.  At  this  point  we  are  not  sure  this  is 
because  of  the  novelty  of  the  tool  or  if  people  really  prefer  to  use  the  HUD. 


FIGURE  5:  HUD  Spectrum 


Figure  5  above  shows  a  picture  of  the  spectrum  of  light  emitted  by  the  Hughes  HUD.  The 
light  spectrum  peaks  at  608  nm.  Figure  6  below  shows  the  spectrum  of  light  reflected 
from  the  wide  screen  in  the  visual  perception  lab.  The  peak  is  around  540  nm.  The 
image  spectrums  were  not  used  much  in  this  paper,  although  we  intend  to  analyze  the 


light  spectrums  of  the  displayed  images  in  a  future  paper  and  use  the  spectrums  a  s  a 
discrimination  factor  in  the  evaluation  of  enhanced  vision  displays. 


2 


% 

I  - 


1 


FIGURE  6:  Screen  Spectrum 


Conclusions  and  Observations 


These  experiments  were  designed  to  test  the  experimental  tools  and  not  to  provide 
answers  to  a  particular  hypothesis.  However,  from  the  plots  in  Figure  1  through  Figure  4  it 
is  clear  people  use  the  HUD  that  displays  IR  imagery.  This  is  no  doubt  due  in  part  to  the 
novel  nature  of  the  HUD.  However,  many  people  who  participated  in  this  test  have 
mentioned  that  they  prefer  to  look  at  the  HUD  because  it  provides  a  clear  image  and 
keeps  their  attention. 

Future  tests  will  address  the  question  of  subject’s  reactions  to  scenes  requiring  driver 
actions.  Using  eyetracker  data  we  will  be  able  to  trace  the  drivers  visual  response  to  cues 
displayed  on  a  visual  aid  display  and  in  the  “real  world”  of  the  180°  wrap-around  screen. 


References 

1. CIE  Technical  Report  on  Fundamentals  of  the  Visual  Task  of  Night  Driving,  1st  edition  1992. 

2.  McCarley,  J.S.,  Krebs,  W.K.  (in  review).  "Visibility  of  road  hazards  in  thermal,  visible,  and  sensor-fused 
nighttime  imagery."  Applied  Ergonomics. 

3.  Sinai,  M.J.,  and  Krebs,  W.K.  (in  review).  "An  assessment  of  sensor-fused  imagery:  a  comparison  using 
target  detection,  target  recognition,  situational  awareness,  and  scene  recognition  tasks."  Journal  of  Applied 
Experimental  Psychology: 

4.  Applied  Science  Labs  Eyetracker  manual,  1996,  Applied  Science  Labs,  Norwood,  MA,  pp.4-7-  to  4-8. 


